Add cross-platform code coverage tooling + CI wiring#990
Draft
johnml1135 wants to merge 3 commits into
Draft
Conversation
The existing /EnableCodeCoverage MSBuild flag (Build/SetupInclude.targets) depends on a VS-Enterprise-only "Code Coverage" data collector and silently fails to collect on Community/Build Tools installs (confirmed live: vstest.console.exe reports "Unable to find a datacollector"). Add coverlet.collector instead, which registers the cross-platform "XPlat Code Coverage" collector and works under plain vstest.console.exe on any VS edition. Wire a `-Coverage` switch into test.ps1 that collects coverage (per-project TestAdapterPath resolved from the centrally-pinned package version, same pattern as the existing NUnit adapter lookup) and renders a summary/HTML report via a local ReportGenerator dotnet tool (.config/dotnet-tools.json). Add the fieldworks-test-coverage skill (how to check that new/changed lines are actually exercised, not just that a test file exists nearby) and reference it from execute-implement/verify-test and the base rubric's test_coverage criterion. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add -Coverage to the CI test.ps1 invocation and upload the rendered summary/HTML report as a build artifact. Have test.ps1 run `dotnet tool restore` before invoking ReportGenerator so a fresh checkout (CI runners, new clones) doesn't just warn and skip the report because the local tool manifest was never restored. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
coverlet.collector's XPlat Code Coverage data collector instruments every module with a matching .pdb next to the test assembly. FieldWorks test projects share one Output/<Configuration> folder rather than per-project bin folders, so with the default MaxCpuCount=0 (parallel testhosts), two testhosts raced to instrument/restore the same shared product DLL (e.g. XMLViews.dll), failing CI with "being used by another process". Force MaxCpuCount=1 in a coverage-only runsettings clone so coverage runs execute test assemblies serially. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/EnableCodeCoverageMSBuild flag (Build/SetupInclude.targets) depends on a VS-Enterprise-only "Code Coverage" data collector and silently fails to collect on Community/Build Tools installs (confirmed live:vstest.console.exereportsUnable to find a datacollector with friendly name 'Code Coverage').coverlet.collectorinstead, which registers the cross-platform "XPlat Code Coverage" collector and works under plainvstest.console.exeon any VS edition.-Coverageswitch totest.ps1that collects coverage (resolving coverlet'sTestAdapterPathfrom the centrally-pinned package version, same pattern as the existing NUnit adapter lookup) and renders a summary/HTML report via a local ReportGenerator dotnet tool (.config/dotnet-tools.json).-CoverageintoCI.yml's test step and upload the rendered report as a build artifact.test.ps1now runsdotnet tool restorebefore invoking ReportGenerator so a fresh checkout/CI runner doesn't just warn and skip the report.fieldworks-test-coverageskill (how to check that new/changed lines are actually exercised, not just that a test file exists nearby) and reference it fromexecute-implement/verify-testand the base rubric'stest_coveragecriterion.Test plan
.\build.ps1— full solution build, clean.\test.ps1 -TestProject "Src/Common/FwUtils/FwUtilsTests" -Coverageon this branch (built against plainmain, not any feature branch) — 375/375 tests passed, coverage collected and summary/HTML report rendered correctly/EnableCodeCoverageflag does not work on a VS 2022 Community install (motivates the coverlet switch)🤖 Generated with Claude Code
This change is